Estimating Components in Finite Mixtures and Hidden Markov Models
نویسندگان
چکیده
When the unobservable Markov chain in a hidden Markov model is stationary the marginal distribution of the observations is a finite mixture with the number of terms equal to the number of the states of the Markov chain. This suggests estimating the number of states of the unobservable Markov chain by determining the number of mixture components in the marginal distribution. We therefore present new methods for estimating the number of states in a hidden Markov model, and coincidentally the unknown number of components in a finite mixture, based on penalized quasi-likelihood and generalized quasi-likelihood ratio methods constructed from the marginal distribution. The procedures advocated are simple to calculate and results obtained in empirical applications indicate that they are as effective as current available methods based on the full likelihood. We show that, under fairly general regularity conditions, the methods proposed will generate strongly consistent estimates of the unknown number of states or components. Some key words: finite mixture, hidden Markov process, model selection, number of states, penalized quasi-likelihood, generalized quasi-likelihood ratio, strong consistency.
منابع مشابه
Estimation of Generalized Multisensor Hidden Markov Chains and Unsupervised Image Segmentation
This paper attacks the problem of generalized multisensor mixture estimation. A distribution mixture is said to be generalized when the exact nature of components is not known, but each of them belongs to a finite known set of families of distributions. Estimating such a mixture entails a supplementary difficulty: One must label, for each class and each sensor, the exact nature of the correspon...
متن کاملEstimating the Number of Components in a Mixture of Multilayer Perceptrons
BIC criterion is widely used by the neural-network community for model selection tasks, although its convergence properties are not always theoretically established. In this paper we will focus on estimating the number of components in a mixture of multilayer perceptrons and proving the convergence of the BIC criterion in this frame. The penalized marginal-likelihood for mixture models and hidd...
متن کاملA Method of Moments for Mixture Models and Hidden Markov Models
Mixture models are a fundamental tool in applied statistics and machine learning for treating data taken from multiple subpopulations. The current practice for estimating the parameters of such models relies on local search heuristics (e.g., the EM algorithm) which are prone to failure, and existing consistent methods are unfavorable due to their high computational and sample complexity which t...
متن کاملLearning High-Dimensional Mixtures of Graphical Models
We consider unsupervised estimation of mixtures of discrete graphical models, where the class variable corresponding to the mixture components is hidden and each mixture component over the observed variables can have a potentially different Markov graph structure and parameters. We propose a novel approach for estimating the mixture components, and our output is a tree-mixture model which serve...
متن کاملLearning Mixtures of Tree Graphical Models
We consider unsupervised estimation of mixtures of discrete graphical models, where the class variable is hidden and each mixture component can have a potentially different Markov graph structure and parameters over the observed variables. We propose a novel method for estimating the mixture components with provable guarantees. Our output is a tree-mixture model which serves as a good approxima...
متن کامل